1. Field of the Invention
The present invention relates to a motion vector detecting device favorably used in an image coding apparatus according to MPEG (Moving Picture Experts Group), and more particularly to an improvement for reducing a hardware volume.
2. Description of the Background Art
To transmit or store signals representing an image, i.e., image signals, having a large amount of data, an image compression technique for reducing the amount of data is essential. The image signal, in general, includes considerable redundancy resulting from correlation between adjacent pixels and human perceptional characteristics. The image compression technique for suppressing the redundancy to reduce the amount of data is termed high-efficiency coding. In the present specification, to avoid useless complicate description, an image signal representing an image and an image itself are equally represented as xe2x80x9cimagexe2x80x9d if not leading to confusion.
As one of such high-efficiency coding systems known is an inter-frame predictive coding system. In this inter-frame predictive coding system, a predictive error, which is a difference between a current image (frame or field) to be coded and a reference image (frame or field) to be referred to, being the preceding or following data in terms of time, is calculated for each pixel of the same position. The predictive error as calculated is thereafter used for coding.
In this system, it is advantageously possible to perform a coding of an image having small movements at high efficiency, due to high correlation between frames (fields). As to an image having large movements, however, errors are disadvantageously increased due to small correlation between frames (fields), leading to an increase in amount of data to be transmitted or stored.
Further, as a system to solve the aforementioned problem known is a motion compensated inter-frame (or inter-field) predictive coding system. In this system, prior to calculation of the predictive error, a motion vector is calculated by comparison of pixel values between the current image and the reference image. The motion vector refers to a vector representing magnitude and direction of an inter-frame (or inter-field) local movement of an image.
A local image in the reference image is moved in accordance with the motion vector as calculated, and then used as a predictive image for the calculation of the predictive error. Specifically, the predictive error is calculated by comparing the predictive image obtained by motion-compensating the reference image with the current image. Then, the predictive error as calculated is used for the coding. The motion vector along with a coding signal corresponding to the predictive error is transmitted or stored as an image signal after compression.
Thus, the motion compensated inter-frame (or inter-field) predictive coding system has an advantage of achieving a high compression efficiency of an image, even if having a large movement, since the inter-frame (or inter-field) predictive coding is performed by adopting the motion compensation using a motion vector.
The detection of the motion vector (predictive image) in the inter-frame (or inter-field) predictive coding system, however, requires enormous amounts of operations, as shown in the document (Mitsubishi Denki Gihou: Vol. 69, No. 3, pp. 12 to 16 (1995), xe2x80x9cA Motion-Estimation LSI with Half-Pixel Precisionxe2x80x9d) and as a result, there is a problem of requiring enormous amounts of hardware to implement a wide-range search of motion vectors.
The present invention is directed to a motion vector detecting device for searching a predictive block included in a reference image, for which a predictive error should be obtained by comparison with a template block included in a current image, and detecting a motion vector indicating a displacement between the template block and the predictive block. According to a first aspect of the present invention, the motion vector detecting device comprises: an operation unit for calculating estimation values each indicating an overall similarity of pixel values between the template block and a displacement block every time when the displacement block is moved in a search window after skipping pixels of the template block and the displacement block, the displacement block having the same size as the template block and included in the search window, and the search window being defined in the reference image as a region including a block corresponding to the template block; a comparison unit for searching a minimum value among the estimation values on the template block and outputting the motion vector corresponding to a displacement block providing the minimum value, which is defined as the predictive block; and an input unit for supplying the pixel values of the template block and the pixel values of the search window to the operation unit.
According to a second aspect of the present invention, in the motion vector detecting device of the first aspect, the input unit comprises a filter unit for performing a filtering of the current image and the reference image and supplying the pixel values of the template block and the pixel values of the search window to the operation unit.
According to a third aspect of the present invention, in the motion vector detecting device of the first aspect, the input unit comprises a filter unit for performing a filtering of the current image and the reference image in an on/off selectable manner in response to an externally-received control signal and supplying the pixel values of the template block and the pixel values of the search window to the operation unit.
According to a fourth aspect of the present invention, in the motion vector detecting device of the second aspect, the filter unit performs the filtering by converting a pixel value of an object pixel to be filtered into a weighted mean of pixel values of a plurality of consecutive pixels along a horizontal scanning-line direction including the object pixel, with an weight to be added to pixels outside a block including the object pixel set zero.
According to a fifth aspect of the present invention, in the motion vector detecting device of the first aspect, the operation unit comprises a template-block storing unit for storing pixel values of all pixels included in the template block after skipping; a search-window storing unit for storing pixel values of all pixels included in the displacement block; an estimation-component-value operation unit for calculating estimation component values for all the pixels included in the template block after skipping, each of the estimation component values indicating a similarity between the pixel values stored in the template-block storing unit and the pixel values stored in the search-window storing unit on a pixel-by-pixel basis; a summation unit for synthesizing the estimation component values to calculate the estimation values, the search-window storing unit comprises search-window storing elements arranged in a matrix, the search-window storing elements being so cascaded and each storing a pixel value for one pixel as to move the displacement block in the search window by storing the pixel values of the search window in order, and the input unit supplies the pixel values of the template block to the template-block storing unit and supplies the pixel values of the search window to the search-window storing unit.
According to a sixth aspect of the present invention, in the motion vector detecting device of the fifth aspect, the template-block storing unit comprises a (n, m)-th template-block storing element for storing a pixel value for one pixel, where n=1 to N (N is an even number) and m=1 to M (M is an even number), the search-window storing unit comprises a (k, m)-th search-window storing element for storing a pixel value for one pixel, where k=1 to 2N and n=1 to M, in a plurality of columns of search-window storing elements and (1, m)-th to (2N, m)-th search window elements are cascaded in this order in any m-th column, the estimation-component-value operation unit comprises a (n, m)-th operation element for calculating an estimation component value for one pixel, where n=1 to N and n=1 to M, the operation unit further comprises a (n, m)-th selection element, where n=1 to N and n=1 to M, the (n, m)-th selection element selects a pixel value stored in either a (2n=1, m)-th or (2n, m)-th search-window storing element in response to an externally-received selection signal, and the (n, m)-th operation element calculates an estimation component value for one pixel indicating a similarity between a pixel value stored in the (n, m)-th template-block storing element and a pixel value selected by the (n, m)-th selection element.
According to a seventh aspect of the present invention, in the motion vector detecting device of the sixth aspect, the summation unit comprises a first summation element for calculating sums of the estimation component values calculated by the (n, m)-th operation element, where n=1 to N/2 and m is an odd number; a second summation element for calculating sums of the estimation component values calculated by the (n, m)-th operation element, where n=1 to N/2 and m is an even number; a third summation element for calculating sums of the estimation component values calculated by the (n, m)-th operation element, where n=N/2+1 to N and m is an odd number; a fourth summation element for calculating sums of the estimation component values calculated by the (n, m)-th operation element, where n=N/2+1 to N and m is an even number; first and second selectors for switchably selecting one and the other of the sums calculated by the second and third summation elements, respectively, in response to an externally-received selection indicating signal; a first adder for calculating sums of the sums calculated by the first summation element and the sums selected by the first selector; a second adder for calculating sums of the sums calculated by the fourth summation element and the sums selected by the selected selector; and a third adder for calculating sums of the sums calculated by the first and second adders, the comparison unit calculates the motion vector by regarding the sums calculated by the third adder as the estimation values and outputs the motion vector, and further searches a minimum value among the sums calculated by the first adder and outputs a vector indicating a displacement between a sub-displacement block providing the minimum value and a sub-template block as another motion vector, and searches a minimum value among the sums calculated by the second adder and outputs a vector indicating a displacement between a sub-displacement block providing the minimum value and a sub-template block as still another motion vector.
According to an eighth aspect of the present invention, in the motion vector detecting device of the fifth aspect, the template-block storing unit comprises a (n, m)-th template-block storing element for storing a pixel value for one pixel, where n=1 to N (N is an even number) and n=1 to M (M is an even number), the search-window storing unit comprises a (n, k)-th search-window storing element for storing a pixel value for one pixel, where k=1 to N and m=1 to 2M, in a plurality of columns of search-window storing elements and (1, k)-th to (N, k)-th search window elements are cascaded in this order in any k-th column, the estimation-component-value operation unit comprises a (n, m)-th operation element for calculating an estimation component value for one pixel, where n=1 to N and n=1 to M, the operation unit further comprises a (n, m)-th selection element, where n=1 to N and n=1 to M, the (n, m)-th selection element selects a pixel value stored in either a (n, 2mxe2x88x921)-th or (n, 2m)-th search-window storing element in response to an externally-received selection signal, and the (n, m)-th operation element calculates an estimation component value for one pixel indicating a similarity between a pixel value stored in the (n, m)-th template-block storing element and a pixel value selected by the (n, m)-th selection element.
According to a ninth aspect of the present invention, in the motion vector detecting device of the eighth aspect, a first summation element for calculating sums of the estimation component values calculated by the (n, m)-th operation element, where n=1 to N/2, m is an odd number and n=1 to M; a second summation element for calculating sums of the estimation component values calculated by the (n, m)-th operation element, where n=1 to M/2, m is an even number and n=1 to M; a third summation element for calculating sums of the estimation component values calculated by the (n, m)-th operation element, where n=N/2+1 to N, m is an odd number, and n=1 to M; a fourth summation element for calculating sums of the estimation component values calculated by the (n, m)th operation element, where n=N/2+1 to N, m is an even number and n=1 to M; first and second selectors for switchably selecting one and the other of the sums calculated by the second and third summation elements, respectively, in response to an externally-received selection indicating signal; a first adder for calculating sums of the sums calculated by the first summation element and the sums selected by the first selector; a second adder for calculating sums of the sums calculated by the fourth summation element and the sums selected by the selected selector; and a third adder for calculating sums of the sums calculated by the first and second adders, the comparison unit calculates the motion vector by defining the sums calculated by the third adder as the estimation values and outputs the motion vector, and further searches a minimum value among the sums calculated by the first adder and outputs a vector indicating a displacement between a sub-displacement block providing the minimum value and a sub-template block as another motion vector, and searches a minimum value among the sums calculated by the second adder and outputs a vector indicating a displacement between a sub-displacement block providing the minimum value and a sub-template block as still another motion vector.
According to a tenth aspect of the present invention, in the motion vector detecting device of the fifth aspect, the template-block storing unit is defined as a first template-block storing unit, the estimation-component-value operation unit is defined as a first estimation-component-value operation unit, the summation unit is defined as a first summation unit and the comparison unit is defined as a first comparison unit, and the device further comprises: second to j-th (jxe2x89xa72) template-block storing units each having the same configuration as the first template-block storing unit; second to j-th estimation-component-value operation units each having the same configuration as the first estimation-component-value operation unit; second to j-th summation units each having the same configuration as the first summation unit; and second to j-th comparison units each having the same configuration as the first comparison unit, any j-th (j=1 to J) template-block storing unit can store pixel values of all pixels included in one template block after skipping, any j-th estimation-component-value operation unit calculates estimation component values for all the pixels included in the one template block after skipping, each of the estimation component values indicating a similarity between the pixel values stored in the j-th template-block storing unit and the pixel values stored in the search-window storing unit on a pixel-by-pixel basis, any j-th summation unit calculates the estimation values on the one template block whose pixel values are stored in the j-th template-block storing unit by synthesizing the estimation component values calculated by the j-th estimation-component-value operation unit, the comparison unit searches a minimum value among the estimation values calculated by the j-th summation unit and outputs a vector indicating a displacement between a displacement block providing the minimum value and the one template block whose pixel values are stored in the j-th template-block storing unit as a motion vector on the one template block, and the input unit supplies pixel values of j template blocks which are different in position to the first to j-th template-block storing units, respectively, and defining a region including the j template blocks as the search window, supplies pixel values of the search window to the search-window storing unit.
Since the device of the first aspect calculates the estimation values between the template block and the displacement block using the pixels after skipping, it is possible to cut the hardware volume of the device.
Since the device of the second aspect calculates the estimation values based on the filtered pixel values, there is no great deterioration in detection accuracy of the motion vector even thought the pixels are skipped.
Since the device of the third aspect can perform the filtering in an on/off selectable manner, there is no need for separately providing a device performing the filtering and a device not performing the filtering and a kind of device can be used for multiple purposes.
Since the device of the fourth aspect performs the filtering by using weighted mean of pixel values of a plurality of consecutive pixels along the horizontal scanning-line direction i.e., the main scanning-line direction on a block-by-block basis, it is possible to simply constitute the input unit and further cut the hardware volume. Furthermore, it is possible to cut the time required for the filtering.
In the device of the fourth aspect, the move of the displacement block is achieved with a simple constitution of matrix of the search window storing elements. The template-block storing unit stores the pixel values of the template block after skipping while the search-window storing unit stores the pixel values of the search window for one displacement block without skipping. Therefore, it is possible to calculate the estimation value on the pixel values after skipping every time when the displacement block is moved in a range of one pixel. In other words, also in this respect, deterioration in detection accuracy of the motion vector due to skipping can be effectively suppressed.
In the device of the sixth aspect, since the estimation-component-value operation unit selects the pixels of the displacement block to be compared with the pixels of the template block in response to the selection signal, it is possible to change a skipping pattern. In other words, a device with a wide range of use is achieved.
Since the device of the seventh aspect can obtain the motion vector on the template block, and further obtain the motion vector on either the odd or upper template block as another motion vector and the motion vector on either the even or lower template block as still another template block in response to the selection indicating signal, an image coding device with high degree of freedom is achieved.
In the device of the eighth aspect, since the estimation-component-value operation unit selects the pixels of the displacement block to be compared with the pixels of the template block in response to the selection signal, it is possible to change the skipping pattern. In other words, a device with a wide range of use is achieved.
Since the device of the ninth aspect can obtain the motion vector on the template block, and further obtain the motion vector on either the odd or upper template block as another motion vector and the motion vector on either the even or lower template block as still another template block in response to the selection signal, an image coding device with high degree of freedom is achieved.
Since the device of the tenth aspect performs the simultaneous operation on a plurality of template blocks by sharing the search-window storing unit, it is possible to cut the time required for operation while suppressing an increase in hardware volume.
One object of the present invention is to provide a motion vector detecting device capable of reducing the hardware volume.
These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.